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ABSTRACT 
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The paper deals v;ith a class of multi- 
dimensional stochastic control problems with 
noisy data and bounded controls encountered 
in aerospace design. The emphasis is on sub- 
optimal design, the optimality being taken 
in ouadraticnean sense. To that effect the 
problem is viewed as a stochastic version of 
the Lurie problem known from nonlinear con- 
trol, theory. The main result' is a separation 
theorem (involving a nonlinear Kalman-like 
filter) suitable for Lurie-type approximati- 
ons (Theorem 30) . The theorem allows for 
discontinuous characteristics. As a bypro- 
duct we prove the existence of strong solu- 
tions to a class of non-Lipschitzian stocha- 
stic differential equations in n dimensions. 
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with suitable matrices C and Q , and a vec- 
tor valued Gaussian white noise 
independent of f . 

The objective is to suppress the effects of 
turbulence by choosing an admissible control 
such that 


(4*) 


d< 


normal acceleration )di 


■ Min 

In order to become amenable to modern 
techniques, the above problem can be easily 
recast within the framework of the following 
canonical problem formulation: 


I PROBLEM FORMULATION 

In order to motivate as well as justify 
the problem formulation (1) — ( 4 ) to be dealt 
with in this paper, lot us start by noting 
the following particular case encountered in 
flight control design. The longitudinal mo- 
tion of an aircraft in turbulence can be 
described (at some level of approximation) 
by the equations 

* A X t + ' m St » 

(1 ‘ ) £ 4 » m Sat (v t <- cj . o c * < r > 

where X^ is a state vector (made of the angle 
of attack, pitch attitude and pitch rate), 

£ is the elevator deflection (scalar) , * 
y t is the control variable (scalar) , the 
state noise representing turbulence is 
assumed Gaussian with the spectral density 
k,/(n>*+ A* ) ; A is a matrix, b, f are 

vectors, and Sat(.) denotes the standard sa- 
turation function. 

The admissible control policies are to be 
based on the observation data only: 

•*#•> (3' ) V ( c 5 {"feedback functionals “of** 1 *’' 

t^S t SSl] } * 

"where the data l'f t ] are generated by ” 
the equations 


(1) dX t - (AX t +• b(4) dt + Fd«; , 

(2) <0; * a* at <3 d«4 , oa^r«p ) 

(3) {MJ e % s {{Uj : U t adapted 

HI £ h } , 

(4) J a f E(X’QX Jds —► Min 

• 5 5 U £ V. 

v;here X* denotes a new state vector, U 4 
a new control variable, m is a vector valued 
Wiener process, 4,F,C,G,<5 are suitable mat- 
rices, b a vector, M a positive bound. 

It is implicit that we work on an underlying 
probability space {S2 t 43, P) with elements de- 
noted by w ; the symbol in (30) denotes 
the d- -algebra of events from A3 generated 
by the data process { Y s , S£i) . 

Also the following technical conditions are 
assumed throughout: 

X* is Gaussian, with Co vX p >0, and 
is independent of 

(5) <5<3* ■ I -» Y, * 0 i 

14 is independent of fWJ , s,d>t}. 

Finally we underline that the solutions to 
stochastic differential equations arising in 
the above formulation and below are required 
to be taken in the strong sense , i.e. causal 
functionals of the given noise sample. The 
.absence of strong solutions rules out the 




TP7-F 








- 


-* r 



feedback solutions to the control problem. 

Hotstlonal remark. The generic variablus 
are suppresse<r*from the notation; the 
symbol I swans the identity matrix; * means 
the transpose; E (. ) , E(.|.) denote the expec- 
tation, reap, the conditional expectation; 
the notation for random processes fw*.j > (u t ) , 
etc., is often shortened to just W, U, etc. 


original problem, i.e. 


Min J 
UtU 


In this case 

A* * 

(13) % 
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II SEPARATION 

The separation principle has been so far 
a key tool for solving stochastic control 
problems with noisy observation data. It 
enables us to transform the latter problem 
into the one with complete observations of 
the state « the separated problem (see for 
instance [Won 1), [Fie]). In this section 
we improve on the separation result of [Ru2] 
by relaxing the assumption Dim X » Dim Y , 
Det C M in (1),(.2). This assumption has 
been a subject of concern in the literature 
on separation (see [Won l],[DaU, where an 
approximate method of dealing with this prob- 
lem has been suggested) . By doing this we 
at the same time establish a basis for the 
development in Section IV. 

In what follows we use the .notation con- 
sistent with [Ru 2], Working with (1) - (5), 
let{x 4 J , {?*! , *2.0 be defined by 

(6) - AX* tit + FdW* , X. * X. , 

' df* - CX t dt + <5 d% , £ - 0 1 

and let 14 be the set of controls 

( 7 ) V. * {{u*| : M* adapted &* , 

IUJ £ M J • 

The definition of the separated problem is 
as follows: 

(8) Definition ( Separated problem correspon- 

ding to (1) - (4) ) . 

(9) - (AX^bL/Jdt + 5C*dz£. X # -E{A.1» 

{z; , j . . . the innovation (Wiener) 
process corresponding to X,t g 

(10) 3 " jE(\*QX)ds — ► Min. i 

the matrix function P t is given by 

P t - A£ + £ A* * FF*- , £ * Cdv . 


We have the following separation theorem 

(11) Theorem . Consider (1) - (10). 

(i) Assume the pair (C,A) is observable 
Then 

(12) Min J £ Min. J 

Ucti U01L 

‘* h (ii) If 1 moreover is the optimal 
control for the separated problem, and if it 
is adapted to the -algebra 

- fip =- rf 

where Vf^l denotes the data corresponding to 
then {U mt \ is also optimal for the 


The proof follows from Lemma 2.2 and Theorem 
3.1 of [Ru 2] in the case Dim X * Dim Y, 

Det C ? O.The latter condition was needed to 
justify the following implication: 

Y u 

For a ^ -adapted control let 

(14) CE(\| fcj) ** C E(i! 4 l(i^) ^.p.l 

then .s ~ 

(15) E(XJfcJ) S E(X t l£>;) w.p.l. 

This implication, however, can be seen to 
hold under the much weaker assumption of ob- 
servability, as shown in the APPENDIX I. 


Ill SEPARATED PROBLEM 

Theorem (11) calls for an optimal solu- 
tion to the separated problem ( 8 ) • This re- 
latively simple Markovian problem has been 
approached by many techniques; none of them, 
however, produced a solution, except for the 
case when Dim *= 1 (or equivalent) . 

Our approach has been via the sample path 
optimality conditions, [Ru 2): 

(16) Theorem . 

(i) The optimal control to the se- 
parated problem exists. 

(ii)Assume Ut ft is such that w.p.l 

(17) E( l? ( * ^QX^dsJft*)#^ a-e.t 
Then U is optimal iff 

(18) u* »-««*/« 2* I 

(19) Special case : Dim X * 1 . Using the 
theorem it has been shown in [Ru 1] that the 
optimal U is given by 

( 20 ) • — k signu*n X t 

and that the resulting Stochastic DE has a 
strong solution. This control moreover satis- 
fies the separation theorem of the previous 
section ( [Ru 2]), and is therefore optimal 
for the correspondine one-dimensional version 
of (1) - (4). 

(21) Multidimensional case: bang-bang cha - 

racter of optimal control. This lesser result 
follows directly from the above Theorem (16) 
by imposing ^essentially the controllability 
of the pair (A,b) . We need to show that with'* 1 
probability 1, on any interval of 

positive length. To Ibis end let us expand 
the expression for as follows: 

x t - e At E(x,)+ jy < 4 * 4 ) feU s + • 


* 




('pen a substitution into (17) , am' ursinn the 
I roportior of stochastic integrals f \;o obtain 

2? » b*K(t' C M le** Fi^ci z\ + 

+ absolutoly continuous path, 


where )t(t) is given by 

r AfU-i) A C3-t), 

K(() ■ ie Q c ds 

4t 


If C is square ami nonninnular, then tho 
stochastic integral above behaves as a sca- 
led down b.rownian notion. If in addition 
! *K(t)o>:pht is not identically aero on any 
interval e* positive length, then the whole 
first torn on the right in a scaled down 
brownian eat!, and conscruentcly has the 
desired property, how 

b* K(l) e M * 0 =» bV(O r S 0 *► 
sp k>* K(i) b *0 bl(ds*0 

=» fqe A % *0-5» 

zp k>, Ab , • . , A"' 1 b e Mspxe(Q) • 


Vho latter ran not happen if.o ? n and xf 
the "air (A,! ) in controlla! le. It is likely 
that the condition on C could he relaxed; 
but we dont pursue this any further. 

Of course the lack of more specific cha- 
racterization of the structure of optimal 
control law inhibits the uscfullncss of the 
separation theorem (11) . As we have noted 
above, other approaches have not fared better. 
We mention at this point 


(22) Martingale based optimality criterion 

oTTem. 7Da ?r 

(23) Comparison theorem method of [Wa] ; 

(24) Bayes formula approach of [Ba] ; 


obtain a quadratic-form-type approximation to 
the solution of the Bellman equation. Although 
the statistical linearization is difficult to 
justify and the approximation used for solving 
Bellman equation is rather crude, the computa- 
tional experience and experiments with simple 
examples seem favorable. 

A direct utilization of the statistical line- 
arization for Lurie-typo optimal stochastic 
control has been reported in tLim) . 

The Lurie-type policies ere also strongly 
suggested by taking a second look at the equ- 
ation ( 1 * ) of the aircraft problem statement; 
with V(t) « k*X 4 (k being a vector) and f « 

« 0, (1 * ) become the well known Lurie system 

(28) ^«AX t fb< t , * k 5at ( - O . 

whose stability has been extensively studied 
(see (Lef)). Some simple examples as well as 
some rather deep investigations (see [Won 2)) 
indicate that a good stable system design of 
the unperturbed mode of a stochastic system 
with complete observations will produce a 
good stochastic control (quadratic Mean sense). 

The aim of this paper is now to answer 
the following : 

Consider the separated problem (9), (10) cor- 
responding to the original problem (1) - (4) . 
Let {U t ] denote a sub-optimal Lurie-type 
control process for the separated problem; 

U fc - € ^ 

such that 

| J(U) - Mfn J j < £ * 

■k 

(i) Does this control law realize a 
feedback , i.c. does the cor- 

responding stochastic DE have a strong solu- 
tion? 

(ii) If so, is this control admissible 
for the original problem (1) - ( 4 ) , i.e. 
is U e U ? 

(iii’t If so, can we estimate the approxi- 
mation error 


(25) Dynamic programming , [Won 3) . 


The above considerations as well as the 
experience with the corresponding determini- 
stic control problem point to a rather com- 
plex nonlinear structure of optimal control 
in more than one dimension .thereby sugges- 
ting to turn to approximations which could 
be handled mathematically and which also 
would be compatible with the nature of our 
control problem; 


/ J(U) “ J 

These questions are answered in the followi ng 
section. 


IV SEPARATION THEOREM FOR LURIE-TYPE 
APPROXIMATIONS 


ons 


Let us denote by $ the class of functi- 


(28) Sub-optimal control policies . 

We will consider Markovian control policies 
of the form a „ 

(27) 4 - * r CR » 

where p({) are functions like Sati , Sign$ , 
and similar. They are simple enough „to instru*. 
ment and to“deal with mathematically. 

A sub-optimal design of these *Lurie-type" 
controls is described in [Won 3), the opti- 
mality being taken in quadratic mean sense. 

The approach combines the dynamic programming 
ant) statistical linearization techniques to 


§ 3 ( gp(£) : r'-vr 1 : 'SPCI) > 

(29) <jp nondecrc* s,n 9 




The separation theorem below is the main re 
suit of this section ^ * 

(30) Theorem . Consider the stochastic 
control problem (l)-(4) and the related sepa- 
rated problem (9) , (10) , Let 

«■ 

(31) 


14 - 
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■x* 


* 


for km 95 * and a vector f (• means 
transpose) . Assume further 


(32) 
or 

(33) 

Then 

(34) 


(C,A) observable if $ ie Lipschitz 

Det C fl 0, (DimX-DimY) ,if <p is 

discontinuous . 


and hence v 

X t s X* * E(XJGj). 


(35) If U is £ -optimal in (i.e. for 
the separated problem) , it is also 
€ -optimal in *U. (i.e. for the original 

control problem) . 


(1) Proof o f UzM . The inclusion follows 
readily by showing that the I to equation (9) 
admits a strong solution. If is Lipschitz 
then there is no problem and the standard Zto 
theory applies. Otherwise we have the follo- 
wing lemma, which is of interest on its own: 


£ is well defined. Next for U e K we 
have that (see (2.23) of £Ru 2]) 

dZt - dY* - dt 

(42) « dr t - c^dfc , 

and hence is also a solution to 

(43) dX t - (A -^C*C ) X* dt f b^Cf***) +• 

4£C*dY t • 

We will now use this equation to verify that 
X fc is in fact adapted £« . This then will 
imply that u t ■ jp(^* # X 4 ) belongs to *U . 

Zf <p is Lipschitz, then fS± -adaptedness is 
immediate. Otherwise ££ -adaptedness readily 
follows from pathwise uniqueness of solutions 
to (43), and that is shown the same way as in 
(i) above ( we note that the lemma above re- 
mains in force for f * f(t,X) such that the 
properties (39) hold uniformly). The rest of 
the argument is a technicality which we dele- 
gate to APPENDIX 3. 


(36) Lemma. Consider the stochastic (Zto) 
differential equation 

(37) dX^ c ( f(X 4 ) + g(Xt))ctt + f- dV5£. 

i € (o,Tj > 

where Dim X 4 » n, |w t J is an n-dimensional 
Wiener process, 

(38) g: R n is a measurable, bounded and 

monotone mapping , i.e. 

a > 

(jV) f: R% r" is Lipschitz and satisfies 
the growth condition 

|| f (X) U £ const • 0 4- 1*1) , 

(40) F 4 is an (nx n) -matrix function, 

nonsingular and continuous on (0,T). 

Then (37) has a strong solution. 

For proof see APPENDIX 2. 


(iii) Proof of (35) is immediate from the 
inequality (12) of the theorem (11) : 

T.ja#* tl® Imam ennk 


Let Oj be such that 
J(U f ) - 

By (1?) 


Min 

<L 


< e 


and so 


Mfr J £ Min J 

<u m 


JCU £ ) 


Min J <: J(U £ )- Min J ± £ . 

iX 


V SUMMARY 

Assume the hypothesis of Theorem (30). 
The sub-optimal stochastic control of Lurie 
type for the control system (1) - (4) is ge- 
nerated by the following feedback structure: 


In order to apply this lemma, we may assume 
b ■ - T* in (9) , (otherwise we take a nonsin- 
gular transformation X ■ KX such that 
K*Xb ■ - jr ) . It is easy to verify that’ 

9 IX) =■ -b<p(b*X) 

is monotone; _ _ 

-(X-X)*b [?Cb*X) 

« - (x'b - 6 

£ 0 

Clearly f (X) - AX and \ - P fr C* also satisfy 
the conditions of the lemma (see the hypothe- 
sis of the theorem) . 

Ill) Proof of um^i. consider again (9) , (31h 

( 41 ) * (a£ +• b<p(^*X t )) dt 4 T>C*dZJ . 

"• * 

By (i) above there is a unique solution X« A 
adapted the ^-algebra and so U 4 « flf* X*> 


dx t - (AX 4 4 bu t )dt + F <?w t 
dY t - CXjdt + G dW t 

dX t - [(A - P t C*C)X t 4 bfif Xt)]dt 4 

♦ P t C*d^ 

P t - AP t 4 P t A* ♦ FF* - P t C*CP t 

- ® < r* ** > 

The choice of p , y is based on the perfor- 
mance of the auxiliary hypothetic system 
(separated system in whichfzjj is a Wiener 
process) 

dtX 4 - [AX t ♦ bfljr* X t )Jdt 4 P t C«CdZ*. 

-■ , ■■ ^ rkj •“*** 

X* has the interpretation of the best mean 
square estimate of the state X^ given the past 
data Yj ,s 5 t. The performance of the system 
is at least as good as that of the separated 
system. 


ORIGINAL PAGE IS 
OF POOR QUALITY . 


APPENDIX 1 

Fix • v > o and 1st ua take the condi- 
tional expectation given the o'-algebra 
in (14); noting the relatione (aee (Ru 2) 

(2. 30), (3. 7), (3. 8)) # w 

lu*l € *w. * at tftfe & Y t ji fi£. 

where JL Beans independence and 

40*’ a & lzl-Z‘ u , j » 

let us evaluate first the left hand side in 
U4> « 

c • 

« c e( e**E0(,)+ i e A H.cX | flj ) 

’’.aV.).,* " 




The right-hand side of (14) becomes next 

, CE(E(<,l«' r .‘’)l^‘!) • Ce(\l<c£’>' 
- c s(e A V sV^^I 


- i«r). 

Thus (14) becomes 

Next, taking derivatives with respect to t, 
and putting t * X afterwards, we obtain 

C ( 6(* e (6£) - E(X r lftf)) » c , 

CA ( d«ro ) “ ^ > 


CA°-'( 


dtto 


) « O . 


Therefore (IS) follows if 

Pdrvk [C,CA, * • • » CA n J *• D'"<ri A > 

that is (Observability . 

APPENDIX 2 ( Proof of (36) ) 

The existence of strong solutions will 
be verified via a device from [Ya] , namely 
by showing that 

(a) a weak solution exists 

and * 

(b) the path-wise uniqueness holds. 

(For simplicity we take X 0 - 0.) 

(a) Recall that by a weak solution of (37) 
we mean a probability_space 

{iS, #5, P, _ 

and a pair of ^processes { ' Such that 

ix*l is ^-adapted, continuous path, 

( { is a Wiener process , 

m dx t - f(X t )dt + g(X t )dt + ^ dW t ,w.p.l, 

all t. 


(A Strong solution of (37) is a sample-path 
continuous" process {X*] adapted and such 
that (37) holds w.p.l.) 

The existence of weak solution is now shown 
as follows. Define 

K ’ f p s ’ 

is clearly 43* -adapted and moreover 

E (**p[ i\ - OA) £mM) * 1 
as a consequence of ?> being bounded and f 
satisfying the growth condition (39) . 
Therefr-'a we can use the Girsanov theorem and 
conclu v that there is a new Wiener process 
{ { on the probability space 

**) r - , 
dp s dP exp [SjdW-i S Il4»!| dt] 


such that 


W t - i% eifi » \Xt t . 


The last equation can be rewritten as 

dW h 

or _ 

(P t dW t -) dX t -[f(V + }t S (Xj]di 

which is (44) . 

(b) While weak existence follows essentially 
from the growth of f and boundedness of g, 
the path-wise uniqueness is a consequence of 
f being Lipschitz and g being monotone. 

Assume that on some probability space 
fo.) we have two pairs of processes (X^And 
(Xt> ^ such that X„ » X. - O.and (w Ay is 
a Wiener process. Define * 

z t - x t - V 

Then is differentiable and we can write 

* (». -f.l’Cf -f c \) -f (\)J +[ 9 ( *,) - 9 < *i)] ) 

Now by the Lipschitz property of f 

* c mst 

and by monotonicity of g const ? 0, 

< V**)* (g(K>~g(x*)) o . 

Hence , . » j 

£ HZJ A const !|ZJI ,112,11 =0 

Thus Z ^ £ O for all t w.p.l and path unique- 
ness follows. 


APPENDIX 3 Y 

(Proof that{x 4 {of (43) is adapted^ in 
the case of non-Lipschitz Cf . ) 


From (ii) of the proof of Theorem (3Q), 
path-wise uniqueness for (43) holds. 

Let J2>* denote the set of all possible paths 
of { x t { restricted to the interval (0,t) and 
denote ® r . Then 


® * ^c n (o,r) ) * & cT 


Co,*) > 
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where we have denoted by r j the 

9* -algebra of Borel sets of the space of 
n-diraensional continuous functions on (0,T) 
resp. (0,t) - C n ( 0,T) resp. C n (0,t). 

Define the mappings 

f : 2) <♦ C n (o,T) 

ft : P & -> C n (o,i) 

by 

f($)(s) -= VV /f(A-gc*ic)x B , - y Cr^)}* 

Clearly 


h 


Jt 


<0,i) 


where denotes the restriction to 

C n (0,t) C n [0,t). By uniqueness, f and f^ 

are one-to-one mappings w.p.l, and hence the 
inverses f *' , f"' exist and 

- - ,-r' 


c 


} m It D 

Next f and f^ are measurable 

resp. for a11 hence by the 

Kuratowski theorem of measure theory the in- 
— *" are measurable. 


verse mappings f 


Let JC 4 denote the coordinate mapping; then 
'*(£) : R 1 * C n <°, T) -w R 1 

is measurable & A ( JP'is the ff-algebra 

of Borel sets on (0,T)), and moreover 

*"'(#) » X±Xio,t.,f"( i) 

so' that 

JC t f m, (£) is measurable • 

The proof is finished by taking 

i- n,i j k - 
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